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Abstract 

Efforts to determine the prevalence of serious emotional disturbance in preschool-aged children 
have been hampered by the lack of a validated measure. The Preschool and Early Childhood 
Functional Assessment Scale (PECFAS) is a multi-dimensional measure that assesses the 
psychosocial functioning of children aged 3-7 years. The concurrent validity and reliability of the 
PECFAS were assessed in a sample of 30 preschool-aged children in a large Head Start program 
in Ventura, California. PECFAS ratings based on in-depth interviews were significantly related to 
parental ratings that the children had mental health problems, psychiatric diagnoses, teacher 
ratings of the child’s need for mental health evaluations, teacher ratings of behavior problems on a 
standardized screening inventory (DIAL-R), and actual referrals for mental health evaluations. 
Interrater reliability for the total PECFAS score was high (r = .90) as was internal consistency of 
the five subscales (alpha = .86). Using the PECFAS scores as a standard, the weighted prevalence 
of serious emotional disturbance in this West Coast Head Start program was 17%, at the lower end 
of the current estimated rate of SED for older children in low income samples (18-26%). 
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Epidemiological studies report rates of childhood psychosocial problems which vary from 
3% to 30% depending on definition, socioeconomic status, measures, and ages of the 
children in question (National Institute of Mental Health, 1990; Schwartz-Gould, Wunsch- 
Hitzig, & Dohrenwend, 1981; Simonian, 1991). The Center for Mental Health Services 
(CMHS) has recently proposed the definition that children with serious emotional 
disturbance (SED) are those between the ages of birth to 18 who have had a diagnosable 
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mental, behavioral, or emotional disorder of sufficient duration to meet diagnostic criteria 
that results in functional impairment that substantially interferes with or limits the child’s 
role or functioning in family, school, or community activities (Federal Register, 1993). 

Based on this definition, a panel of experts convened by CMHS reviewed all published 
studies and concluded that the prevalence of SED in children aged 9 to 17 years in the 
general population is in the range of 9-13% (Friedman, Katz-Leavy, Manderscheid, & 
Sondheimer, 1996). 

The rate of SED appears to vary greatly by socioeconomic group. One of the best estimates 
of the prevalence of SED in low income children is based on an epidemiological study in 
Puerto Rico (Bird, Yager, Staghezza, & Canino, 1990), that reported a prevalence of 18%for 
children who had a psychiatric diagnosis and a score of 60 or lower on the Children’s Global 
Assessment Scale (CGAS). Based on this and other studies (Costello et al., 1996; Zahner, 
Pawelkiewiez, DeFrancesco, & Adnopoz, 1992), the CMHS report estimated that the 
prevalence of SED in poor children is twice the rate found in the general population and 
therefore in the range of 18-26%. 

In the search for measurements of psychiatric impairment to bring consistency to mental 
health research, the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 
1989; 1994) was developed as a measure of degree of impairment in children aged 6-16 and 
has achieved increasing use in mental health research (Hodges, Doucette-Gates, &Liao, 

1999; Hodges & Gust, 1995; Hodges & Wong, 1996, 1997; Hodges, Wong, & Latessa, 
1998). In contrast to the Children’s Global Assessment Scale (CGAS), another commonly 
used functional assessment scale (Bird, Canino, Rubio-Stipec, & Ribera, 1987; Green, Shirk, 
Hanze, & Wanstrath, 1994), the CAFAS comprises a set of written items describing 
behavior that are organized into domains of functioning. Within each domain, behaviors are 
grouped into levels of impairment: severe, moderate, mild, and no or minimal impairment. 
For each scale, the rater determines the level of impairment that best describes the youth’s 
most severe level of dysfunction during the time specified by the user (e.g., last month, last 
3 months). The scores assigned to each of the categories are as follows: 30 for severe, 20 for 
moderate, 10 for mild, and 0 for minimal or no impairment. 

The 1989 version of the CAFAS, consisted of five scales for the youth and two for the 
caregiver. The youth scales are as follows: Role Performance (i.e., how effectively the youth 
fulfills societal roles in school, home, and community), Behavior toward Self and Others 
(i.e., appropriateness of the youth’s daily behavior), Moods/Emotions (i.e., modulation of 
the youth’s emotional life). Thinking (i.e., ability of the youth to use rational thought 
processes), and Substance Use (i.e., youth’s substance use and the extent to which it is 
inappropriate or disruptive). The total score refers to the sum of the five scales assessing the 
youth, with a range from 0 to 150. A higher score reflects greater impairment. The current 
version of the CAFAS (Hodges, 1994) has eight youth scales (School Role Performance, 
Home Role Performance, Community Role Performance, Behavior toward Others, Mood, 
Self-Harmful Behavior, Substance Use, and Thinking) and includes a list of strengths and/or 
goals appropriate to each scale (Hodges, 1997; Hodges, 1998). 

The assessment of functional problems in children younger than school-age is less well 
defined than it has been in older children, although preliminary research suggests that the 
prevalence of psychosocial problems in preschool-aged children may be as high as in older 
children (Favigne et al., 1993). Favigne and his colleagues reported a prevalence rate of 
13% positive screens on a behavior problem questionnaire with preschool-aged children, 
virtually identical to the rate found with school-aged children (14%) from the same 
community using the same instrument (Costello, Edelbrock, & Costello, 1988). 
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In recent years, the potentially greater impact of earlier interventions has led to a federal 
mandate to include screening for psychological and developmental problems in all 
Medicaid-covered well-child pediatric visits and for increased federal funding for agencies 
that serve preschool-aged children (Knitzer, 1993). Although there are many developmental, 
cognitive, and psychosocial screening instruments targeted at this age range (Achenbach, 
1992; Behar, 1977; First & Palfrey, 1994), the lack of a validated scale for rating the overall 
psychosocial functioning of younger children has precluded a precise estimate of the 
prevalence of serious emotional disturbances in preschool-aged children (Friedman et al., 
1996). Hodges (1994a) has recently created a revised measure, the Preschool and Early 
Childhood Functional Assessment Scale (PECFAS), that permits the assessment of children 
aged three through seven years. The current study presents preliminary validity data for the 
PECFAS. 

Validating a psychosocial assessment instrument for preschool-aged children presents a 
number of challenges: younger children have a shorter developmental history, a narrower 
range of daily expectations, fewer areas of experiences outside the home, and more limited 
verbal skills compared to school-aged children (Little, Jellinek, & Murphy, 1994). Since 
there are no gold standard measures that rate psychosocial functioning for preschool-aged 
children, the current study employs multiple informants and multiple measures to establish 
the concurrent validity of the new instrument (Lavigne et al., 1993; Pilkonis, Heape, & 
Ruddy, 1991). We assessed the validity of the PECFAS as a measure of overall functioning 
for preschool-aged children by examining its relationship to ratings of parents and preschool 
teachers on different measures. Then, using PECFAS scores as a benchmark, we estimated 
the prevalence of SED using a large sample of preschool-aged children from low-income 
families. 


METHOD 

Subjects 

The study took place in Ventura, CA and was conducted in collaboration with Child 
Development Resources of Ventura County, California Head Start program (CDR), which 
serves a large population of parents and children from predominantly low-income, Hispanic 
backgrounds. As an addition to a required developmental screening during the month of 
September 1994, all the parents of children who were enrolled in CDR were asked to 
complete a brief checklist assessing the child’s behavioral and emotional problems. 

The study was conducted in the seventeen centers and 55 classrooms of CDR which had a 
combined enrollment of 952 children during September of 1994. Two hundred thirty-three 
(24%) parents were not approached for a variety of reasons, the most common of which was 
that the parents did not attend the developmental screening at the scheduled time when the 
parent advocates were present. Fifty-one of the 717 approached parents (7%) declined to fill 
out a screening checklist, leaving a total questionnaire sample of 663. 

Procedure 

While Head Start teachers conducted developmental screening with the children, parent 
advocates asked parents to fill out a number of registration forms as well as a brief checklist 
(the Pediatric Symptom Checklist; Jellinek et al., 1986) and to give consent for a possible 
later interview. Participation was voluntary. Parents who indicated consent were scheduled 
for semi-structured interviews at CDR by the senior author or Head Start supervisors. The 
research design called for interviews with the parents of 40 children who scored above the 
cutoff on the PSC and the parents of 20 children who scored below the cutoff. Due to 
logistical problems, only thirty interviews could be scheduled. Interviewers, Head Start 
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supervisors, and teaching staff were blind to the initial screening scores on the PSC. All 
procedures and consent forms were approved by the Massachusetts General Hospital 
Subcommittee on Human Studies. 


Measures 

Demographics — Demographic information including child gender, age, and racial/ethnic 
background were assessed from items on the PSC form. Insurance status was determined 
through the Head Start enrollment database. 

The Pediatric Symptom Checklist — The Pediatric Symptom Checklist (PSC) is a 35 
item questionnaire which has been used most often with school aged children (Jellinek, 
Murphy, & Burns, 1986; Jellinek et al., 1998). Several studies have also established the 
validity of the PSC with preschool aged children (Little et al., 1994; Pagano et al., 1995). 

For children aged three through five years, a slightly different scoring procedure has been 
used: the four PSC items which relate to school functioning are dropped and the total score 
for the instrument is based on the 31 non school related items. A cutoff score of 24 or higher 
is considered to indicate the presence of psychosocial problems. This method of scoring the 
PSC was followed in the current study. Four new items had been pilot tested on the PSC 
form used in our study, but since these new items did not markedly add to the accuracy of 
the original instrument, they were dropped from all subsequent analyses in the current study. 
The PSC form in our study also included an additional item which asked whether the parent 
thought the child had a problem with behavior or emotions for which the child needed help. 
Forms were available in English and Spanish. 

Preschool and Early Childhood Functional Assessment Scale (PECFAS) — The 

PECFAS is an interviewer generated rating which is based on five sub-scales which assess 
the child and two additional scales which can be used to assess the child’s environment 
(Hodges, 1994). Although PECFAS ratings can be made from any source of data about the 
child (like clinical interviews) the author of the PECFAS had also developed a semi- 
structured interview for parents about the child (Hodges, 1994b). This interview was 
administered in our study. Both Spanish and English versions of the interview were 
available and parents were interviewed in the language of their choice. 

The child’s functioning is rated in seven areas: Role Performance (i.e., how effectively the 
child fulfills societally defined roles in home, in preschool/day care settings, and in the 
community);Thinking/Communication (i.e., the ability of the child to use rational thought 
processes); Behavior Toward Others/Self (i.e., appropriateness of child’s daily behavior); 
Moods/Emotions (i.e., modulation of the child’s emotional life); and Self-Harming Behavior 
(i.e., extent to which the child can cope without resorting to self-harming behavior or 
verbalizations). 

For each of the subscales, multiple items of severe, moderate, mild, and no problems are 
provided so that the rater can categorize the degree of impairment. For each subscale, the 
degree of impairment is assigned a score of 30, 20, 10, or zero, based on the specific items 
endorsed. The five area scores which assess the child's functioning are summed to provide 
an overall functioning score ranging from 0-150. Two additional scales are available to 
assess the caregiver but do not affect the child’s overall score: Basic Needs (i.e., caregiver’s 
ability to provide for basic needs of the child including food, shelter, clothing, medical care, 
and safety) and Family/Social Support (i.e., the degree to which the quality of the 
relationships and the guidance and nurturance provided within the family meet the child’s 
developmental needs). 
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For older children a total CAFAS score of 40 has been considered indicative of severe 
emotional disturbance based on a five-scale sum total score using the 1989 version of the 
CAFAS (Friedman et al., 1996). A total score of 40 on the PECFAS was tentatively used to 
indicate SED for younger children. 

PECFAS ratings were given by the interviewers immediately after the interviews. These 
scores were checked by experienced research assistants who read through the interviewers’ 
notes and made independent ratings. Although these ratings usually agreed with the 
interviewers’ original PECFAS scores, the few instances of disagreement that were noted 
were discussed until a consensus was arrived at. These consensus PECFAS scores were 
reported in the analyses that follow. In order to calculate inter-rater reliability for this study, 
another research assistant reviewed the written interview notes and made an independent 
rating. The PECFAS scores of Rater 2 were contrasted with the scores established by Rater 
1/Consensus. 

Parent-Rating of Child Psychosocial Problems — The first cross validator of the 
PECFAS ratings was the parents’ responses on the PSC screening questionnaire. In addition 
to the recommended cutoff score of 24 for the 31 item form, parents’ answers to the single 
question, “Does your child have any emotional or behavioral problem for which she/he 
needs help?” were used as another cross validator of the PECFAS ratings. Similar single 
questions asking parents whether their children had a problem for which they needed help 
have been used in studies of low-income children (Bird, Gould, Rubio-Stipec, Staghezza, & 
Canino, 1991; Zahner et al., 1992) and in the authors’ studies of children in the same 
California county (Murphy et al., 1996; Pagano et al., 1995). Both ways of assessing 
parental concern over child psychosocial functioning were used to validate the PECFAS. 

Psychiatric Diagnoses — Data on Attention Deficit/Hyperactivity Disorder (ADHD) and 
Oppositional Defiant Disorder (ODD) were collected for all interview subjects using these 
modules from the Schedule for Affective Disorders and Schizophrenia in School-Age 
Children — Parent report (K-SADS-P), a structured psychiatric interview that assesses DSM- 
IV diagnoses. Only these two disorders were assessed because in the authors’ previous work 
with low income children (Murphy, Reede, Jellinek, & Bishop, 1992) and in Lavigne’s 
study of preschoolers (Lavigne et al., 1993), they accounted for more than two-thirds of all 
the clinically significant diagnoses given to preschoolers. 

Of the three most widely used structured psychiatric interviews available at the time of the 
study, none had been validated for use with preschool-aged children. The K-SADS-P was 
chosen because it involves a simple rendering of DSM-IV symptoms and can be 
administered by lay-interviewers. Even though the K-SADS-P interview has not been 
validated for use with children younger than six years, both of the diagnoses are valid for 
children in this age range. 

Teacher-Rating of Child Psychosocial Problems — As part of our study, the senior 
author met with all of the Head Start teachers during the middle of the school year and asked 
them to review their class rosters and to list any children whom they felt were in need of 
evaluation for mental health services. The Head Start teacher ratings of children with 
psychological problems who needed further evaluation were compared to the PECFAS 
ratings. 

The Developmental Indicators for Assessment of Learning-Revised (DIAL-R) 

— The Developmental Indicators for Assessment of Learning-Revised (DIAL-R) is a well 
established and validated educational screen for children between the ages of 2 and 6 years 
(Barnett, Faust, & Sarmir, 1988; Glascoe, 1991; Mardell-Czudnowski & Goldenberg, 1984). 
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Three scales assess potential problems in the areas of motor skills, concepts, and language 
skills, and a fourth scale summarizes the teacher’s observations of behavioral problems 
during testing and the possible need for further social/affective assessment. The entire 
DIAL-R is completed in 20-30 minutes and requires minimal training for administration and 
scoring. For each of the four subscales, cut-off scores indicate whether the child is 
functioning adequately or has a potential problem in that area (Mardell-Czudnowski & 
Goldenberg, 1990). Only the social/affective observations are reported here in comparison to 
PECFAS scores. 

Mental Health Referrals — A third indicator of the PECFAS ’s validity was actual referral 
to a mental health clinician over the course of the school year. CDR had a full time licensed 
social worker who was available for all mental health referrals. Referrals to the mental 
health clinician for evaluations and/or therapy were recorded by the CDR Head Start health 
service coordinator and these data were made available to the research team at the end of the 
year. 

Data Analysis 

In the current study, between-group comparisons on categorical variables were made with 
the chi square test. On the interval data, differences in means between the two groups were 
tested using analysis of variance (ANOVA). Statistical significance was defined as a two- 
tailed alpha of p < .05. 


RESULTS 


As a whole, 9% (59/663) of the sample had a PSC score of 24 or higher. Five hundred-fifty 
of the 663 parents (83%) indicated on the PSC form that they would be willing to be 
interviewed. As a result of the intended two-to-one over sampling of highly symptomatic 
children, 19 children with PSC scores of 24 or higher and 1 1 children with PSC scores of 23 
or lower were interviewed. Forty-two parents had to be contacted to obtain 30 completed 
interviews (71%). 

Demographic Factors and PECFAS Scores 

As shown in Table 1, the questionnaire sample of 663 children was 51% (336/663) male 
with a mean age of 3.8 years (s.d. = .5). The majority of the children were 4 years old (73%), 
Hispanic (77%), and covered by Medicaid (77%). Compared to official program data on the 
952 children enrolled on 10/01/94, the obtained sample of 663 children did not differ from 
the remainder of the Head Start population in terms of gender, age, ethnicity, primary 
language of parent, or type of insurance coverage. 

The interview sample consisted of 20 boys and 10 girls. All 30 of the subjects were between 
3-5 years of age, with a mean age of 3.8 years (s.d. = 1.4). Sixty-three percent (19/30) of the 
interview sample were from Hispanic backgrounds, 27% (8/30) were from Anglo 
backgrounds, 7% (2/30) were from Asian backgrounds, and one child (3%) was African- 
American. Seventy-three percent (22/30) of the interview sample were from minority 
backgrounds. Eighty-three percent (25/30) of the children were from families with Medicaid 
health insurance. The interview sample did not differ significantly from the full sample in 
terms of gender, age, ethnicity, or health insurance. 

The mean PECFAS score for the interview sample was 26.0 (s.d. = 20.1) and 8 of the 30 
children (27%) had a total score of 40 or higher on the PECFAS. Using a PECFAS score of 
40 or higher as the cut off, 75% of the PSC positive screens (7%of the total questionnaire 
sample) were found to be true positive cases, and 1 l%of the PSC negative screens (10% of 
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the total questionnaire sample) were found to be false negative cases, yielding a weighted 
prevalence of serious emotional disturbance in this West Coast Head Start program of 
17%just below the lower end of the current estimated rate of SED for older children in low 
income samples (18-26%). 

Neither mean nor categorical PECFAS scores were significantly related to child gender, age, 
ethnicity, or insurance status. There was a non-significant trend for children from a minority 
background to have higher mean PECFAS scores (30.0, s.d. = 20.2 vs 15.0, s.d. = 16.0; F = 
3.6, df= l,p<.10). 

Parent-Rating of Child Problem and PECFAS Scores 

As shown in Table 2, 16% (108/663) of the questionnaire sample and 33% (10/30) of the 
interview sample of parents stated that they thought their child had a problem with behavior 
or emotions for which the child needed help. Children with a parent-rating of a psychosocial 
problem were significantly more likely to be PECFAS cases than children with no parent- 
rating of a problem (50% vs 15%; x2 = 4.2, df = 2, p<.05; SN= .85, SP= .50). As would be 
expected, children with parent-rating of a psychosocial problem also had significantly higher 
mean PECFAS scores (mean = 38.0, s.d. = 16.0) than children with no parent-rating of a 
problem (mean = 20.0, s.d. = 15.0; F = 6.2, df = 1, p < .01). 

Psychiatric Diagnoses and PECFAS Scores 

Thirty percent (9/30) of the interview sample were classified as having Oppositional Defiant 
Disorder (ODD) and 17% (5/30) were classified as having Attentional Deficit Hyperactivity 
Disorder (ADHD). Forty-three percent (13/30) were classified as having either ODD, 

ADHD, or both. Children with ODD were significantly more likely to be PECFAS cases 
than children without ODD (56% vs 14%; x2= 7.3, df = 2, p < .05; SN= .86, SP= .56) and to 
have significantly higher PECFAS mean scores (mean= 42.2, s.d. = 20.2 vs mean = 19.0, 
s.d. = 15.0; F= 11.3, df = 1, p < .01). Children with ADHD were twice as likely to be cases 
on the PECFAS and to have higher mean PECFAS scores than children without ADHD, 
although these findings failed to reach statistical significance. Children with either ODD, 
ADHD, or both diagnoses were significantly more likely to be PECFAS cases than children 
without any diagnoses (75% vs 32%; x2 = 4.5, df = 2, p < .05; SN= .88, SP= .46) and to 
have significantly higher PECFAS mean scores (mean = 38.5, s.d. = 22.1 vs mean = 16.5, 
s.d= 12.2; F = 12.2, df = 1, p < .01). 

Teacher-Rating of Child Problem and PECFAS Scores 

Of the 663 children in the questionnaire sample, 653 children were still enrolled when the 
teachers rated their class lists for psychosocial problems. Ten percent of these children 
(62/653) were rated as needing mental health evaluations. In the interview sample, 20% 
(6/30) of the children had teacher-rating of a psychosocial problem. Children with a teacher- 
rating of a psychosocial problem were significantly more likely to be PECFAS cases than 
children with no teacher-ratings of a problem (67% vs 17%; x2 = 4.1, df = 2, p < .05; SN = . 
83, SP = .67). Children with a teacher-rating of a psychosocial problem had significantly 
higher mean PECFAS scores (mean = 41.7, s.d. = 23.1) than children with no teacher-rating 
of a problem (mean = 22.1, s.d. = 13.4; F = 6.9, df = 1, p < .01). 

DIAL-R and PECFAS Scores 

Six percent (33/579) of the total sample and 33% (10/30) of the interview sample were rated 
by Head Start teachers as having potential problems in social/emotional development on the 
DIAL-R at the beginning of the school year. Children who were rated with potential 
problems in social/emotional development were significantly more likely to be PECFAS 
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cases than children who were not rated with potential problems (56% vs 14%; x2 =5.5, df = 
2, p < .05; SN= .86, SP = .55). Children with potential social/emotional problems had 
significantly higher mean PECFAS scores (mean = 38.9, s.d. = 20.4) than children without 
potential problems (mean = 20.5, s.d. = 16.5; F = 4.4, df = 1, p < .05). 

Referral and PECFAS scores 

Four percent (25/663) of the total sample and 13%(4/30) of the interview sample were 
referred to the licensed social worker for further evaluation and/or treatment during the 
school year. Children who were referred for mental health evaluations had significantly 
higher mean PECFAS scores (mean = 45.0, s.d. = 26.4) than non-referred children (mean = 
23.1, s.d. = 15.4; F = 8.4, df = 1, p < .05). 

Inter-Rater and Internal Consistency Reliability 

Blind to the original PECFAS score by Rater 1/Consensus, a master’s level psychologist 
independently gave a PECFAS rating to each case based on the structured interview protocol 
and notes. The exact PECFAS ratings of the two raters showed a correlation of r = .89 (p < . 
0001). PECFAS score agreement was exact for 29 cases and within 10 points on the 
remaining case (e.g. PECFAS ratings of 30 vs 40). Alpha reliability calculated for the total 
PECFAS score based on the five subscales indicated high internal consistency (alpha = .86). 

DISCUSSION 

Our study suggests that the PECFAS provides a valid and reliable measure of overall 
psychosocial functioning for three to five year old children that will prove to be very useful 
to both researchers and clinicians. Indicators of the validity of the child’s impairment score 
on the PECFAS were provided by (a) the presence of a psychosocial problem or (b) 
psychiatric diagnosis based on parent-report, (c) teacher-ratings of the child’s need for 
psychological evaluation, (d) teacher-ratings of the presence of a potential social/emotional 
problem during developmental testing, and (e) referral to the mental health clinician over the 
course of the school year. Interrater reliability and internal consistency for the PECFAS total 
score was excellent. 

Using a PECFAS score of 40 or higher as the standard, 75% of the PSC positive screens 
(7% of the total sample) were found to be true positive cases, and 1 1 % of the PSC negative 
screens (10% of the total sample) were found to be false negative cases, yielding a weighted 
prevalence of serious emotional disturbance in this West Coast Head Start program of 17%, 
at the lower end of the current estimated rate of SED for older children in low income 
samples (18-26%). 

There are a number of limitations that must be taken into account in interpreting the findings 
of the study. First of all, the interview sample was relatively small and thus susceptible to 
variation due to chance. Secondly, the sample may not have been representative due to loss 
of subjects in the sampling. Thirdly, the children studied were predominantly Hispanic from 
low-income areas, and the prevalence rate of SED may differ in other racial or ethnic groups 
from various income levels in other locations. 

Nevertheless, the findings suggest that the PECFAS will be a valid and reliable scale for 
rating the overall psychosocial functioning of preschool-aged children. The PECFAS’s focus 
on behavioral descriptions of impairment and its depiction of the young child’s functioning 
across domains make it a useful organizing tool for treatment planning among staff and with 
parents as well as providing a way to prioritize which preschool-aged children and their 
families would most benefit from expanded services. This in turn would provide a way to 
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link level of care to level of need (Newman & Hodges, 1995), conduct program evaluation 
and planning (Pokorny, 1991), and analyze costs vs. benefits. 
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